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ABSTRACT 

This  study  addresses  the  problem  of  estimating  the 
development,  procurement,  and  installation  costs  of  surface 
ship  electronic  warfare  equipment  of  the  future.   The  Cost 
Estimating  Relationships  (CERs)  were  developed  using  the 
following  factors:   year  of  development,  weight,  volume, 
sensitivity,  power  output,  gain,  complexity  and  dummy  variables 
for  active  equipment,  equipment  designed  for  large  ships 
and  one  for  those  designed  for  small  ships. 

Cost  estimates  are  made  for  three  systems  presently  under 
development  by  Raytheon  Company  and  Hughes  Aircraft  Company 
under  a  design-to-price  program. 
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I.   INTRODUCTION 

r 
Traditionally  cost  estimates  for  military  equipment  have 

been  derived  through  engineering  techniques.   These  techniques 

are  extremely  time  consuming  and  require  detailed  information 

about  the  proposed  equipment.   It  has  been  found  in  recent 

years  that  estimates  made  using  Cost  Estimating  Relationships 

(CERs)  based  on^equipment  characteristics  or  parameters 

require  less  time.   Some  experts  maintain  that  they  are  at 

least  as  accurate  as  the  engineering  estimates  as  long  as  care 

is  taken  in  choosing  the  CER. 

An  advantage  of  CERs  is  that  since  a  proper  CER  is 
developed  from  historical  data  on  similar  equipment,  it  takes 
into  account  hidden  costs  which  would  be  impossible,  practically 
speaking,  to  separate  out,  and  which  are  perhaps  not  consider- 
ed when  making  engineering  estimates. 

Before  one  begins  estimating  costs  by  the  parametric 
method,  one  must  locate  or  develop  a  CER  which  adequately 
represents  the  equipment  in  question. 

The  objective  of  this  thesis  is  to  develop  such  CERs  to 
be  used  in  estimating  the  costs  for  development ,  procurement, 
and  installation  of  three  design-to-price  electronic  warfare 
suites  currently  under  development  by  Raytheon  and  Hughes 
for  the  United  States  Navy,  and  to  estimate  those  costs  for 
the  proposed  systems. 


The  CERs  are  developed  from  data  presented  in  Ref.  1 
utilizing  BIMED02R,  a  stepwise  linear  regression  program 
developed  by  the  University  of  California  at  Los  Angeles  (UCLA). 
This  program  considers  all  independent  variables  available 
and,  at  each  step,  includes  in  the  equation  the  one  which 
is  most  highly  correlated  with  the  dependent  variable  for 
the  first  step  or  the  one  with  highest  partial  correlation 
with  the  previous  step  for  all  succeeding  steps.   This  process 
may  also  remove  a  variable  if  that  variable's  F-value  drops 
below  the  F-value  to  remove,  designated  for  the  particular 
regression,  as  more  variables  are  added  to  the  equation. 
More  detailed  information  concerning  BIMED02R  is  contained 
in  Ref.  2. 

Section  II  of  this  thesis  concerns  the  assumptions  made 
about  the  data  and  explanations  of  the  various  parameters 
and  variables  available  in  the  data  base. 

The  development  of  the  CERs  is  presented  in  Section  III 
along  with  an  example  demonstrating  what  can  happen  in  linear 
regression  if  the  independent  variables  in  the  equation  are 
highly  inter-correlated.   Also  included  is  an  example  of  the 
use  of  a  graphical  method  of  selecting  independent  variables 
by  Mallows  found  in  Ref.  3. 

The  CERs  developed  in  Section  III  are  used  in  Section  IV 
to  estimate  the  cost  of  the  proposed  systems  and  a  comparison 
is  made  of  those  costs  with  the  ones  estimated  in  Ref.  1. 

The  Appendix  presents  the  data  matrix,  refers  to  its  source 
and  describes  adjustments  that  were  made  to  the  original  data. 


II.   DESCRIPTION  OF  THE  MODEL 

The  data  for  this  thesis  consists  of  the  costs  and 
characteristics  of  21  electronic  warfare  (EW)  systems  developed 
from  1962  to  1974.   These  systems  represents  all  of  the  major 
surface  ship  EW  systems  developed  for  the  U.S.  Navy  during 
that  time  period. 

A.  ASSUMPTIONS 

The  following  are  assumptions  made  concerning  the  data 
used  in  this  thesis, 

1.  All  of  the  figures  in  the  data  taken  from  Ref.  1 
are  correct  except  as  noted  in  the  Appendix.   It  should 

be  pointed  out  that  this  data  differs  greatly  in  some  areas 
from  the  values  found  in  Ref.  4,  however,  such  differences 
can  be  explained  by  what  aggregation  of  equipment  one  is 
referring  to,  such  as,  is  one  interested  in  the  price  of 
a  whole  suite  or  just  the  black  box  without  antennas,  etc. 
The  data  used  in  this  thesis  describes  the  total  system 
for  one  ship. 

2.  Costs  for  the  individual  equipments  are  independent 
of  the  costs  of  all  other  pieces  of  equipment.   It  is  felt 
that  this  assumption  is  valid  since  cost  of  development  of 
follow-on  equipment  which  was  a  modification  of  existing 
equipment  was  adjusted  where  possible  by  Mitchell  and 
Mullins  for  Ref.  1.   Where  such  adjustment  was  found  to  be 
impossible,  those  data  points  were  excluded  from  the  data 
matrix. 

B.  PARAMETERS  AND  VARIABLES 

The  parameters  and  variables  used  in  this  thesis  are 

described  below.   Dollars  are  measured  in  thousands  of  1975 

dollars . 

1.   Development  Cost  (DEVC)  includes  the  cost  of 
Research,  Development,  Test  and  Evaluation  (RDT  &  E)  of 
the  equipment  in  question. 
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2.  Procurement  Cost  (PRDC)  is  the  cost  of  producing 
the  first  unit  of  the  equipment. 

3.  Installation  Cost  (INST)-  is  the  average  cost  of 
installing  the  equipment  on  the  surface  ships  for  which 
it  was  designed. 

4.  Year  ox  Development  (YR)  is  a  measure  of  the  number 
of  years  from  the  base  year  of  1969  to  the  year  of  develop- 
ment.  This  variable  is  negative  for  equipment  developed 
prior  to  1969, 

5.  Weight  (WT)  is  the  weight  in  pounds  of  the  equipment 

6.  Volume  (VOL)  is  the  volume  in  cubic  feet  of  the 
equipment . 

7.  Sensitivity  (SENS)  is  the  receiver  sensitivity  of 
the  equipment  measured  in  decibels,   All  values  are  negative 

8.  Power  Output  (KWO)  is  the  power  output  of  the 
active  equipments  measured  in  kilowatts.   This  variable 
is  zero  for  passive  equipment. 

9.  Gain  (GAIN)  is  the  antenna  gain  measured  in  decibels 

10.  Active  (ACT)  is  a  dummy  variable  with  a  value  of 
one  for  active  equipment  and  zero  for  passive  ones. 

11.  Complexity  (CPLX)  is  a  measure  of  complexity  for 
the  equipment,  in  this  data  base,  ranging  from  one  to  three, 
one  signifying  relative  simplicity  and  three  extreme 
complexity.   The  source  of  this  variable  is  discussed  in 
the  Appendix. 

12.  Large  (LG)  is  a  dummy  variable  with  a  value  of  one 
for  equipment  designed  for  guided  missile  destroyers  (DDG) 
and  larger  ships  and  zero  otherwise. 

13.  Small  (SM)  is  a  dummy  variable  with  a  value  of  one 
for  equipment  designed  for  destroyer  (DD)  and  smaller  ships 
and  zero  otherwise.   For  equipment  designed  for  ships  in 
both  categories,  both  variables  LG  and  SM  are  one. 


C.   FORM  OF  THE  MODELS 

Each  model  developed  in  this  thesis  is  a  linear  regression 
equation  containing  one  dependent  cost  variable,  an  intercept 
term  and  any  number  of  the  independent  variables  described 
above,  numbers  four  through  thirteen  inclusive. 


COST  =  a   +  a„  YR  +  a,  WT  +  . . .  +  a,0  SM 
o     4        o  13 

From  examination  of  the  residual  plots  at  various  stages 

during  the  development,  there  was  no  indication  that  polynomial 

or  other  transformations  should  be  used. 
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III.   DEVELOPMENT  OF  COST  ESTIMATING  RELATIONSHIPS 

With  each  CER,  work  was  begun  by  examination  of  the 
correlation  matrix  and  scatter  plots  of  dependent  versus  each 
independent  variable  for  those  equipments  for  which  data  were 
available.   Next,  the  data  was  fed  into  the  BIMED02R  stepwise 
linear  regression  program.   Regression  analysis  was  done  with 
and  without  a  constant  term,  that  is  without  and  with  a  zero 
intercept.   In  each  case  the  fit  was  considerably  better  with 
a  constant  included;  consequently,  the  zero  intercept  models 
were  discarded.   BIMED02R  may  be  controlled  as  to  the  number 
of  steps  taken  by  designating  a  particular  number  of  steps 
or  by  setting  values  for  F-level  for  inclusion  and  deletion. 
Setting  F-levels  allows  the  variables  to  enter  the  model  one 
at  a  time  as  long  as  their  F-level  to  enter  for  the  individual 
variable  is  higher  than  the  level  designated.   If  adding  a 
new  variable  causes  the  F-level  of  a  previously  entered 
variable  to  decrease  below  the  level  set  for  deleting,  that 
previously  entered  variable  will  be  removed.   In  order  to 
observe  as  much  of  the  interaction  as  possible  during  the 
early  analysis,  no  maximum  number  of  steps  was  specified  and 
F-levels  were  set  at  0.01  for  inclusion  and  0.005  for  deletion 
When  certain  models  appeared  to  be  of  interest,  F-levels  were 
increased  to  more  realistic  levels  and  later  the  maximum 
number  of  steps  was  designated  in  order  to  restrict  the  model. 
During  this  phase,  residuals  were  listed  and  plotted  against 
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the  computer  value  for  the  dependent  variable,  and  each  of 
the  independent  variables.   Each  residual  plot  was  examined 
for  points  that  appeared  to  be  outliers  and  for  any  indication 
of  a  need  for  any  type  of  transformation.   Eventually  a  best 
model  was  selected  to  be  the  CER  for  each  of  the  three  costs. 

In  the  statistics  given  after  each  CER,  the  F  ratio  is  the 
test  value  for  the  hypothesis  that  the  regression  is  not 
significent.   If  the  F  ratio  is  larger  than  the  table  value, 
the  hypothesis  is  rejected  and  the  regression  is  significant. 
The  F  value  for  each  independent  variable  is  used  to  test  the 
hypothesis  that  the  coefficient  of  the  variable  in  question 
equals  zero.   For  instance,  if  at  a  particular  step  there 
are  three  independent  variables  in  the  model,  the  F  value 
for  each  variable  is  a  measure  of  how  much  is  lost  by  assuming 
that  the  model  only  includes  the  other  two  variables.   The 
hypothesis  is  rejected  if  the  table  value  is  less  than  the 
F  value. 

A.   DEVELOPMENT  COST 

Development  costs  were  available  for  twelve  (12)  pieces 
of  equipment.  The  correlation  matrix  for  those  equipments, 
listed  below,  was  examined. 


DEVC 

YR 

WT 

VOL 

SENS 

KWO 

1.000 

0.384 

0.685; 

0.772 

-0.014 

0.299 

DEVC 

1.000 

0.283 

0.387 

0.162 

0.443 

YR 

1.000 

0.870  v 

0.627 

0.640 

WT 

1.000 

0.468 
1.000 

-0.739 

-0.692 

1.000 

VOL 

SENS 
KWO 
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GAIN       ACT        CPLX       LG         SM 


0.213 

0.124 

0.846 

-0.214 

-0.108 

DEVC 

0.251 

0.443 

0.631 

-0.328 

-0.426 

YR 

0.068 

0.511 

0.620 

-0.018 

-0.368 

WT- 

0.080 

-0.609 

-O.S21v 

0.326 

0.113 

VOL 

0.048 

-0.749 

-0.077 

0.075 

0.253 

SENS 

0.172 

0.737 

0.491 

-0.507 

0.082 

KWO 

1.000 

0.067 

-0.003 

-0.014 

0.294 

GAIN 

1.000 

0.423 

-0.507 

-0.098 

ACT 

1.000 

-0.449 

-0.185 

CPLX 

1.000 

-0.577 

LG 

1.000 

SM 

Weight,  volume,  and  complexity  are  highly  correlated  with 
development  costs.   However,  since  weight  and  complexity  are 
also  very  highly  correlated  with  volume,  one  must  insure  that 
if  weight  and  complexity  are  included  in  the  final  model, 
volume  is  not  and  vice  versa.  I 

When  the  data  was  processed  by  BIMED02R,  the  first  four 
steps  brought  in  CPLX,  ACT,  WT,  and  GAIN  in  that  order.   In 
each  step  the  equation  appeared  acceptable;  however,  step 
four  presented  the  best  model.   The  CER  at  that  step  was: 

DEVC  =  -7474.87891  +  .30165  WT  +  142.60353  GAIN  - 
3457.53076  ACT  +  4757.61719  CPLX 


which  had  the  following  statistics: 
.2 


R2  =  .9287 


R 

(adjusted)  is  a  monotonic  increasing  function  with 

a  final  value  of  .888. 

Standard  error  of  estimate  =  1396.3611 

Mean  value  of  the  dependent  variable  =  3802.3 

F  ratio  =  22.782 

Table  value  F  ,    or   ,   0.  =  3.84 
(,  .  do  ,  4  ,  b; 
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F  value  for  each  independent  variable  is:   WT  8.8495 

GAIN  4.5523 

ACT  12.8525 

CPLX  35.1702 

Table  value  F,  nn   1   QN  =  3.46 
( ,  yu,  l,  o; 

This  model  is  statistically  appealing  and  intuitively 
reasonable.   The  negative  sign  on  the  constant  is  acceptable 
unless  one  is  attempting  to  predict  the  development  cost  of 
a  very  light,  low  gain,  simple  piece  of  equipment.   The 
equipment  being  considered  in  this  thesis  is  heavy  and  complex 

Indications  of  trends  in  the  data  over  time  are  that 
future  equipments  will  have  these   same  two  characteristics. 
With  the  independent  variables  WT,  GAIN,  and  CPLX  as  one 
would  suspect,  an  increase  in  the  value  of  any  of  the  three 
will  increase  the  development  cost.   The  negative  coefficient 
of  the  active  dummy  variable  indicates  that  active  equipment 
tend  to  be  less  expensive  to  develop  for  a  given  weight,  gain, 
and  complexity  than  passive  equipment.   This  is  understandable 
when  one  considers  the  different  uses  to  which  active  and 
passive  equipments  are  put.   Active  equipment  is  designed  for 
a  specific  or  a  few  specific  types  of  manipulation  of 
incoming  signals  over  a  small  band  of  frequencies,  then 
retransmission  of  these  signals.   Passive  equipment  on  the 
other  hand  is  used  for  the  evaluation  and  identification  of 
incoming  signals  over  a  very  wide  range  of  frequencies.   The 
processing  in  this  case  must  maintain  the  signal  character- 
istics so  that  accurate  identification  ma,y  be  made. 
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When  residual  plots  were  examined,  neither  outliers  nor 
indication  of  a  need  for  a  transformation  were  found.  \   Many 
transformations  were  tried  however  in  an  attempt  to  improve 
the  model,  but  no  improvement  was  noted. A  There  are  two 
computed  values  for  development  cost  with  negative  signs. 
The  equipment  in  question  in  each  case  is  very  simple  and 
light,  two  hundred  (200)  pounds  in  the  first  case  and 
twenty-five  (25)  pounds  in  the  other,  and  thus  in  a  range  of 
little  concern  in  this  thesis. 


ACTUAL  DEVC 

COMPUTED  DEVC 

RESIDUAL 

SLQ19 

1886.0999 

148.4023 

1737.6975 

SLQ26 

3654.0000 

4915.4297 

-1261.4297 

SLQ28 

11694.0000 

11823.8711 

-129.8711 

SLQ30 

1707.0000 

1807.3008 

-100.3008 

SLQ302 

1600.0000 

1343.2773 

256.7227 

SLQ12A 

1754.0000 

2223.4141 

-469.4141 

SLQ17 

7256.0000 

7289.4141 

-33.4141 

SLQ12 

380.0000 

-90.0703 

470.0703 

WLR1 

3598.0000 

4770.0430 

-1172.0430 

WLR3 

45.0000 

-570.6680 

615.6680 

WLR8 

11694.0000 

9814.2344 

1879.7656 

WLR11 

360.0000 

2153.4727 

-1793.4727 

The  standard  error  of  estimate  of  this  CER  is  very  large,  but 
is  considered  acceptable  in  light  of  the  wide  range  of 
development  costs,  forty-five  thousand  dollars  ($45,000)  to 
eleven  million  six  hundred  ninety-four  thousand  dollars 
($11,694,000),  and  the  small  size  of  the  data  base. 

B.   PROCUREMENT  COST 

1.   Development  Cost  Excluded 

The  data  base  for  procurement  cost  contained  all 
twenty-one  (21)  pieces  of  equipment.   The  correlation  matrix, 
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listed  below,  showed  that  weight,  volume,  and  complexity  are 
very  highly  correlated  with  procurement  costs  and  again  both 
complexity  and  weight  are  highly  correlated  with  volume. 


PROC 
1.000 


YR 

0.542 
1.000 


WT 

0.920 
0.372 
1.000 


VOL 

0.920 
0.492 
0.878 
1.000 


SENS 

0.407 
0.072 
0.513 
0.350 
1.000 


KWO 

0.593  PROC 
0.465  YR 
0.589  WT 
0.642  VOL 
0.519  SENS 
1.000  KWO 


GAIN 


ACT 


CPLX 


LG 


SM 


0.286 

0.271 

0.735 

-0.230 

-0.210 

PROC 

0.169 

0.151 

0.554 

-0.421 

-0.435 

YR 

0.195 

0.320 

0.602 

-0.125 

-0.281 

WT 

0.257 

0.361 

0.787  • 

-0.355 

-0.127 

VOL 

0.144 

0.755 

0.074 

0.036 

-0.307 

SENS 

0.013 

0.477 

0.349 

-0.508 

-0.207 

KWO 

1.000 

-0.122 

0.068 

-0.347 

-0.007 

GAIN 

1.000 

0.328 

-0.258 

-0.208 

ACT 

1.000 

-0.314 

-0.144 

CPLX 

1.000 

-0.085 
1.000 

LG 

SM 

When  regression  analysis  was  performed  on  the  data,  the  first 
four  steps  took  in  variables  WT,  CPLX,  YR,  and  SM  in  that 
order.   Any  one  of  the  steps  furnishes  a  very  satisfactory 
model,  however  step  four  results  in  the  best.   The  fit  in 
this  case  is  exceptional  and  once  again  follows  one's 
intuition . 

The  CER  was: 

PROC  =  -726.24341  +  .22550  WT  +  349.72095  CPLX  + 
67.14648  YR  +  324.85962  SM 
which  had  the  following  statistics 

R2  =  .959 
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R2 

(adjusted)  is  a  monotonic  increasing  function  with  a 

final  value  of  .9487 

Standard  error  of  estimate  =  276.8726 

Mean  value  of  the  dependent  variable  =  956.82 

F  ratio  =  93.651 

Table  value  F  =2.96 

(  •  y*3  >  4 ,  i  / ) 

F  value  for  each  independent  variable  is:   WT  152.7812 

CPLX  4.0917 

YR  10.3247 

SM  5.1110 

Table  value  F,  n_   .   ._.  =  3.03 
(.90;  1,  1/ ) 

As  in  the  previous  CER,  the  constant  has  a  negative  sign  and 
is  acceptable  for  the  same  reasons.   The  positive  coefficients 
for  WT  and  CPLX  are  as  one  would  suspect  since  generally 
procurement  costs  rise  as  the  values  for  those  two  indepen- 
dent variables  increase.   The  positive  coefficient  for  the 
dummy  variable  SM  indicates  that  if  a  piece  of  equipment  is 
being  designed  for  use  on  destroyer  and  smaller  ships,  it  is 
more  expensive  than  those  developed  solely  for  larger  ships. 
Those  units  which  are  designed  for  use  on  smaller  ships  are 
generally  more  compact  and  consequently  more  expensive  to 
fabricate  for  a  given  weight.   Variable  YR  also  has  a  positive 
coefficient  which  can  be  explained  by  the  fact  that  over  the 
years  included  in  the  data  base,  prices  for  procurement  have 
increased  more  than  can  be  explained  by  weight  and  complexity. 

As  can  be  seen  from  the  table  below,  the  computed  first 
unit  cost  for  three  systems  is  negative,  however  since  the 
three  units  concerned  are  of  relatively  low  cost,  they  do  not 
detract  from  the  CER.   It  is  also  interesting  to  note  that 
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two  of  the  three  equipments  with  negative  residuals  are  the 
same  ones  with  negatives  residuals  for  development  cost. 


ACTUAL  PROC 

COMPUTED  PROC 

RESIDUAL 

ULQ5AX 

99.0800 

-45.6897 

144.7697 

ULQ6A 

128.2400 

107.6472 

20.5928 

ULQ6B 

130.6000 

346.9700 

-216.3700 

ULQ6C 

225.2800 

475.0002 

-249.7202 

SLQ191 

935.7998 

1468.3806 

-532.5808 

SLQ192 

588.0198 

717.5596 

-129.5398 

SLQ22 

959.0698 

930.2021 

28.8677 

SLQ23 

961.9900 

900.8877 

61.1023 

SLQ24 

359.6499 

500.4072 

-140.7573 

SLQ261 

2156.3799 

1896.6470 

259.7329 

SLQ262 

1120.7100 

930.7031 

190.0068 

SLQ28 

5226.1367 

5199.8320 

26.3047 

SLQ301 

1304.1199 

1551.7834 

-247.6636 

SLQ302 

821.6099 

773.4519 

48.1580 

SLQ12 

453.0898 

618.9177 

-165.8279 

SLQ17 

2725.0898 

2269.3391 

455.7507 

SLR12 

16.3000 

-275.1497 

291.4495 

WLR1 

79.3000 

336.6997 

-257.3997 

WLR3 

7.4600 

-448.9043 

456.3643 

WLR8 

1613.0198 

1572.1064 

40.9133 

WLR11 

175.0700 

259.1987 

-84.1287 

2. 

With  Development  Co 

st  Included 

Although  the  author  did  not  feel  that  using  develop- 
ment cost,  calculated  by  use  of  the  CER  in  paragraph  A,  with 
the  obvious  uncertainty  involved,  was  a  good  variable  to  be 
used  in  explaining  the  cost  of  procuring  the  first  unit,  he 
did  feel  that  after  a  piece  of  equipment  had  been  under 
development  for  a  while  a  more  accurate  estimate  of  develop- 
ment cost  would  be  available  and  could  be  used  for  that 
purpose.   For  that  reason,  regression  analysis  was  performed 
using  the  twelve  pieces  of  equipment  for  which  development 
costs  were  known. 
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Although  no  acceptable  CER  was  found,  an  interesting 
phenomenon  occurred  when  the  stepwise  regression  program  was 
allowed  to  run  with  low  F-level  values  and  is  presented  here 
to  demonstrate  a  problem  concerned  with  high  intercorrelations 
in  a  data  base.   The  equations  resulting  from  the  various 

early  steps  were  unacceptable  due  to  very  low  F  values  for 

2       2 

the  individual  variables.   The  values  for  R   and  R  ,     , .   .  ,. 

(adjusted) 

increased  continually  and  the  standard  error  of  estimate 
continued  to  decrease.   Then  suddenly  at  step  nine,  everything 
fit  together  and  the  F  values  of  all  the  variables  increased 
sharply.   The  following  model  was  the  result  of  that  step: 

PROC  =  2811.59351  +  .05553  DEVC  +  .02824  WT  +  3.81016  VOL  + 
25.98703  SENS  +  73,68718  KWO  +  77.15399  GAIN  - 
1667.51343  ACT  -  1027.33765  LG  -  1718.32324  SM 


which  had  the  following  statistics: 

R2  =  .9997 

R2 

(adjusted)  is  a  monotonic  increasing  function  with  a 

final  value  of  .9973 

Standard  error  of  estimate  =  65.119 

Mean  value  of  the  dependent  variable  =  1291.67 

F  ratio  =  666.08 

Table  value  F/  95.  9   3)  =  8*81 

F  value  for  each  independent  variable:   DEVC 


DEVC 

5, 

,6128 

WT 

4. 

,2697 

VOL 

31. 

,925 

SENS 

67, 

.1465 

KWO 

102 

.7486 

GAIN 

130, 

,7182 

ACT 

62, 

,6125 

LG 

41, 

,8613 

SM 

133, 

.5989 
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Table  value  F,  on   i  Q.  ■  2,68  • 
t , ou;  1,0) 

At  first  glance,  one  would  think  this  CER  was  exceptional;  it 

does  describe  the  data  base  very  very  well  as  shown  by  the 

statistics.   However,  if  one  examines  the  model  itself  and 

attempts  to  explain  the  coefficients,  problems  appear. 

One  would  have  trouble  explaining  why  an  increase  in  the 

sensitivity  of  a  piece  of  equipment  that  is  a  more  negative 

number,  would  decrease  the  price,  and  why  a  piece  of  equipment 

designed  for  use  on  both  a  small  and  large  ship  with  all  other 

variables  constant  would  -be  $1.7  million  cheaper  per  copy  than 

one  designed  solely  for  large  ships.   Obviously  the  signs  of 

the  coefficients  cannot  be  explained  except  by  multicollin- 

earity,  correlation  between  the  independent  variables  in  such 

a  way  that  the  different  variables  interact  in  the  model  to 

balance  each  other,  and  thus  describe  the  particular  data 

base  extremely  well.   It  must  be  remembered  that  the  data 

set  is  small  (12)  and  that  nine  parameters  are  used  in  the 

model.   If  one  were  sure  that  the  intercorrelation  experience 

in  the  past  data  would  also  occur  in  future  equipment,  this 

model  could  be  used  to  good  advantage;  however,  it  is  felt 

that  such  assurance  is  difficult  to  attain  and  should  not 

be  assumed. 

C.   INSTALLATION  COST  ESTIMATING  RELATIONSHIP 

Examining  the  data  base,  one  finds  that  there  is  an 
installation  cost  available  for  nineteen  of  the  equipments, 
however,  since  the  SLQ  22,  SLQ  23,  and  SLQ  24  were  never 
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installed  on  any  ship  and  since  the  estimated  installation 
cost  for  each  were  higher  than  any  other  equipment  in  the  data 
base,  those  three  equipments  were  excluded  from  the  data 
base  used  for  this  CER, 

Study  of  the  correlation  matrix,  listed  below,  and  scatter 
plots  of  the  independent  variables  against  installation  cost 
offered  only  a  preview  of  difficulties  to  come. 


INST 

YR 

WT 

VOL 

SENS 

KWO 

1.000 

0.203 

0.666 

0,618 

-0.338 

0.414 

INST 

1.000 

0.368 

0.449 

-0.317 

0.559 

YR 

1,000 

0.882 

-0.624 

0.615 

WT 

1.000 

-0.518 
1.000 

0.712 

-0.575 

1.000 

VOL 

SENS 
KWO 

GAIN 


ACT 


CPLX 


LG 


SM 


0.107 

0.044 

"   0.603 

-0.119 

-0.157 

INST 

0.176 

0.174 

0.439 

-0.347 

-0.515 

YR 

0.249 

0.296 

0.688 

-0.137 

-0.306 

WT 

0.331 

0.366 

0.829 

-0.389 

-0.137 

VOL 

0.097 

-0.805 

-0.497 

0.158 

0.289 

SENS 

0.022 

0.408 

0.516 

-0.589 

-0.042 

KWO 

1.000 

-0.069 

0.139 

-0.271 

0.097 

GAIN 

1.000 

0.489 

-0.324 

0.022 

ACT 

1.000 

-0.262 

-0.262 

CPLX 

1.000 

-0.455 
1.000 

LG 

SM 

No  variable  was  highly  correlated  with  installation  cost, 
however,  several  independent  variables  were  highly  correlated 
with  each  other. 

When  regression  analysis  was  applied  to  the  data,  another 
interesting  phenomenon  occurred.   First  WT  was  taken  into  the 
model  giving  reasonable  but  poor  statistics.   During  the 
next  three  steps,  CPLX,  ACT  and  SENS  entered,  in  that  order, 
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leaving  an  equation  at  the  end  of  each  step  which  was 
unacceptable  due  to  low  F  values.   On  step  five,  WT  was 
removed  from  the  model  and  the  statistics  improved  consider- 
ably.  The  CER  at  that  step  was; 

INST  =  331.97876  +  254.31705  CPLX  -  356,32886  ACT  + 
8.42499  SENS 


which  had  the  following  statistics 
.2 


R2  =  .593 


R 

(adjusted)  is  a  monotonic  increasing  function  with  a 

final  value  of  .4913 

Standard  error  of  estimate  =  125.42 

Mean  value  of  the  dependent  variable  =  195.86 

F  ratio  =  5.828 

Table  value  F,  OK   „   10N  =  3.41 
( . 95 ;  o ,  13) 

F  level  of  each  independent  variable:   CPLX   9.7667 

ACT    6.7163 
SENS    4 . 3534 

Table  value  F(>9Q.  ^  13)  =  3.14 

The  statistics  for  this  CER  are  acceptable.   As  was  indicated 
in  the  development  CER,  the  passive  equipment  tends  to  be 
more  expensive  to  install  than  active  and  increasing  complexity 
of  a  unit,  of  course,  increases  the  installation  cost.   A 
contradiction,  however,  does  appear  in  the  sensitivity 
coefficient.   Since  the  sensitivity  variable  is  measured  in 
negative  decibels,  the  more  negative,  the  higher  the  sensi- 
tivity; intuition  would  require  that  the  coefficient  have  a 
negative  sign.   In  this  case,  increasing  the  sensitivity 
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decreases  installation  cost.   Examining  the  correlation  matrix 
again,  it  is  found  that  sensitivity  is  highly  negatively 
correlated  with  the  active  variable.   This  is  easily  under- 
stood when  one  realizes  that  passive  equipment  has  high 
receiver  sensitivity  in  order  to  maintain  incoming  signals  at 
sufficient  quality  for  evaluation.   The  assumption  that  this 
negative  intercorrelation  will  remain  true  in  future  equipment 
is  reasonable,  consequently  this  CER  is  acceptable. 

Considerable  investigation  using  various  transformations 
of  variables  made  no  improvement  over  the  above  CER,  so  an 
attempt  was  made  to  find  a  better  CER  by  a  graphical  method 
of  selecting  independent  variables  for  inclusion  by  C.L  Mallows 
described  in  Ref.  3. 

This  method  uses  a  statistic,  C  ,  which  measures  the  sum 

P 

of  the  squared  biases  plus  the  squared  random  error.   C   is 
calculated  by  the  formula: 

C   =  2§|  _  (N  _  2  ) 
P     s2         p' 

where 

RSS  is  the  residual  sum  of  squares 

2 

s   is  a  good  unbiased  estimate  of  the  variance  of  the 

random  error  inherent  in  the  dependent  variable 
N  is  the  number  of  data  points 

p  is  the  number  of  terms  in  the  equation 

2 
Reference  3  determines  an  estimator  for  s   from  an 

equation  with  all  of  the  independent  variables  included.   A 

forty-five  degree  straight  line  is  drawn  on  a  graph  of  number 
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of  variables  included  versus  C   from  the  origin.   This  is 

done  since  for  p  variables  with  negligible  bias,  C   =  p. 

The  distance  from  the  horizontal  axis  to  that  line  is  random 

error  and  above  it  is  bias  error.   At  this  point  a  C   is 

P 

calculated  for  each  possible  equation  containing  one 

variable.   The  equation  with  the  lowest  C   contains  the  first 

P 

variable  to  be  included  in  the  final  model.   Next  C  's  are 

P 

calculated  for  all  models  including  the  first  variable  and 

one  of  the  remaining  variables.   The  equation  with  the  lowest 

C   contains  the  first  variable  selected  plus  the  second 
P 

variable  to  be  included  in  the  final  model.   This  process  is 

continued  adding  one  additional  variable  each  step  until  all 

the  variables  are  included.   At  this  point  there  is  one  best 

equation  for  each  step  from  which  one  must  select  the  best 

model:  in  most  cases  it  is  the  one  with  the  lowest  C   value 

P 

regardless  of  whether  the  point  is  above  or  below  the  line. 
Normally  one  is  willing  to  accept  some  bias  in  order  to 
minimize  variance. 

This  method  was  attempted  using  the  installation  cost 
data  with  the  hope  that  using  C   to  select  independent 
variables  would  give  better  results  than  using  correlation 
as  used  in  stepwise  regression  analysis.   It  was  felt  that 
the  model  with  all  variables  included  did  not  explain  the 

dependent  variable  very  well  and  the  statistic  from  that 

2 
regression  confirmed  it.   That  estimate  of  variance,  s  ,  as 

well  as  estimates  of  variance  derived  from  several  other 

regression  equations  were  used  in  the  formula  for  C  ,  however, 
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none  of  the  results  improved  upon  the.  model  developed  through 

stepwise  regression. 

As  a  matter  of  fact,  since  the  graphical  method  does  not 

allow  for  removal  of  variables,  as  was  used  in  the  step 

regression  analysis,  the  best  model  found  by  the  C   analysis 

was  one  with  WT  and  a  constant  only. 

One  could  use  the  long  method,  calculating  C   for  each 

p 

of  the  2   possible  combinations  of  variables,  but  it  is 
not  felt  that  the  results  would  be  worth  the  enormous  amount 
of  calcualtions  required.   Graph  4  shows  the  best  results 
attained  using  Mallows  method.   The  estimate  of  variance  used 
is  the  minimum  encountered  during  investigations  of  the 
various  models  for  the  installation  CER.   As  can  be  seen,  the 
process  was  only  carried  out  to  four  variables.   At  that  stage, 
two  points  (7,  11,  10)  and  (7,  11,  10,  13),  found  by  stepwise 
regression  but  not  by  the  C   analysis,  were  below  what  should 

ST 

have  been  the  minimum  C   at  each  of  steps  three  and  four.   It 

P 

is  obvious  that  this  method  will  not  be  of  use  on  data  with 
high  intercorrelation  until  a  way  of  removing  variables  is 
included. 
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IV.   COST  COMPUTATIONS  AND  ANALYSIS 

Presently  under  the  Design-to-Price  Electronic  Warfare 
Program  (DTP  EW),  Hughes  Aircraft  Company  and  Raytheon  Company 
are  each  developing  an  electronic  warfare  system  for  the 
United  States  Navy.   These  systems  are  to  have  three  stand- 
alone levels  of  operational  capacity  designated  DTP-1,  the 
least  expensive,  DTP-2,  a  moderately  priced  suite,  and  DPT-3, 
which  includes  DTP-1  and  DTP-2.   Present  contracts  require 
each  company  to  develop  and  build  two  DTP-3  prototypes. 
Production  decisions  will  then  be  made  based  on  competitive 
prototype  evaluation.   Present  plans  are  to  build  one  hundred 
sixteen  (116)  DTP-1,  one  hundred-eighteen  (118)  DTP-2  and 
fifty-nine  (59)  DTP-3. 

Since  the  contract  is  a  design-to-price  type,  prices  have 
been  specified,  and  the  equipment  is  being  developed  to  meet 
those  prices.   The  cost  targets  are  three  hundred  thousand 
dollars  ($300,000),  five  hundred  thousand  dollars  ($500,000) 
and  one  million  four  hundred  thousand  dollars  ($1,400,000) 
respectively  and  are  to  include  comulative  average  production, 
initial  spares,  and  installation  costs.   Translating  cumulative 
average  production  costs  to  first  unit  cost,  the  targets  are 
six  hundred  eighteen  thousand  dollars  ($618,000),  one  million 
thirty-two  thousand  dollars  ($1,032,000),  and  two  million 
two  hundred  thirty-five  thousand  dollars  ($2,235,000). 
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All  price  estimates  in  this  section  were  calculated  using 
up-to-date  parametric  information  received  from  PME  107X  on 
19  February,  1975  as  follows; 

RAYTHEON 


YR 

WT 

SENS 

GAIN 

ACT 

CPLX 

SM 

DTP-1 

6 

2458 

-55 

22 

0 

2 

1 

DTP-2 

6 

3932 

-55 

-22 

0 

2 

1 

DTP- 3 

6 

8331 

-55 

22 

1 

3 

0 

HUGHES 

YR 

WT 

SENS 

GAIN 

ACT 

CPLX 

SM 

DTP-1 

6 

2095 

-63 

20 

0 

2 

1 

DTP-2 

6 

2933 

-63 

20 

0 

2 

1 

DTP- 3 

6 

6036 

-63 

14 

1 

3 

0 

All  cost  estimates  are  in  thousands  of  1975  dollars.   Prediction 
interval  (PI)  are  calculated  for  ninety-five  percent  confidence 
for  each  CER. 

A.   DEVELOPMENT  COST  ESTIMATES 

The  following  development  cost  estimates  were  calculated 
using  indicated  CER  in  column  one  and  two  and  are  the  program 
managers  estimates  as  of  19  February  1975  in  the  last  column. 


RAYTHEON 
HUGHES 

Total 


THESIS  CER 
PI  (+  3220) 

8990.77 
7157.65 

16148.42 


REFERENCE  1  CER 
PI  (+  2406) 

5055.95 
3663.47 

8719.42 


PME  107  X 


16,298 
14,914 

31,212 
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The  CER  from  Reft  1  is; 

DEVC  =  3.52335  +  1,92814  PROC 

PROC.  is  the  first  unit  price  as  determined  by  Ref.  1 
procurement  CER  number  1.   The  estimates  using  CERs  are  much 
lower  prices  than  the  program  manager  estimate.   This  is  due 
to  the  fact  that  instead  of  just  developing  a  system  capable 
of  one  operational  level  as  is  the  case  in  the  data  base, 
DTP-3  must  be  developed  to  operate  at  three  levels.   The 
calculations  of  development  cost  in  Ref.  1  used  a  ninety 
percent  cumulative  learning  for  the  development  of  two  DTP-3 
systems.   Since  they  are  identical,  logic  would  indicate  that 
there  would  be  no  development  cost  for  the  second  unit.   It 
is  interesting  to  note  however,  that  by  applying  the  learning 
curve,  the  CER  from  this  thesis  gives  cost  figures  very  close 
to  those  estimated  by  PME  107X  as  can  be  seen  in  the  following 
table.   The  ninety  percent  learning  was  applied  to  columns 
one  and  two. 

THESIS  CER      REFERENCE  1  CER      PME  107X 

RAYTHEON         16,183.38        9,100.7  16,298 

HUGHES  12,883.77        6,594.24  14,914 

TOTAL  29,067.15       15,694.94  31,212 

B.   FIRST  UNIT  PROCUREMENT  COST  ESTIMATES 

First  unit  procurement  cost  estimates  were  calculated 
using  the  CER  as  indicated  in  columns  one  and  two.   Column 
three  contains  the  program's  manager  estimate  of  first  unit 
procurement  cost  as  of  19  February  1975. 
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THESIS  CER 
PI(  +  584.2) 

RAYTHEON 

DTP-1  1255.22 
DTP-2  1587.55 
DTP-3  2604.44 


REF  1  CER  1/CER2 
PI(±  1080/±1086) 

774-79/839.51 
1238.49/128736 

2622.19/2623>93 


PME  107X 


496.13 

947.20 

2005.32 


HUGHES 

DTP-1  1173.36        660.74/72g  54  513.41 

DTP-2  1362,33        924.34/gg3  g3  880.24 

DTP-3  2086.92       1900 .  44/..  n_  no  2261.10 

IvZo . bo 

Ref.  1  used  two  CERs  to  estimate  procurement  cost,  they  are: 


and 


PROC,  =  -1207.12  +  .314561  WT  -  18.7253  SENS  +  6.62988  ERP 


PROC2  =  92.6764  +  .303836  WT 


ERP  is  effective  radiated  power. 

It  would  be  impossible  to  explain  the  differences  in 
prices  between  the  three  price  estimates  for  each  manufacturer's 
equipment  in  prices  except  to  say  that  each  was  derived  using 
a  different  estimator.   In  the  case  of  Ref,  1,  a  smaller 
number  of  data  point  were  used  in  determining  each  CER  than 
this  thesis.   It  is  interesting  to  note  that  the  CER  developed 
in  this  thesis  estimates  consistantly  $600,000  above  PME  107X 
except  for  the  Hughes  DTP-3  for  which  the  estimate  is  under 
PME  107X. 
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C,   INSTALLATION  COST  ESTIMATE 

The  cost  estimates  for  installation  costs  in  column  one 
and  two  are  calculated  by  the  CER  as  indicated.   Column  three 
contains  the  program  managers  estimates  as  of  19  February  1975 


THESIS  i 

CER 

REF.  1 

CER 

PME  107X 

PI (+270 

.9) 

PI (+318 

.3) 

RAYTHEON 

DTP-1 

377.24 

187,68 

43.15 

DTP-2 

377.24 

239.00 

52.79 

DTP- 3 

275.23 

359.38 

274.83 

HUGHES 

DTP-1 

309.83 

189.65 

56.1 

DTP-2 

309.83 

231.38 

65.2 

DTP- 3 

207.81 

388.48 

145.2 

Ref.  1  used  a  CER  based  on  first  unit  costs  as  estimated  by 
PME  107X;  the  CER  is: 

INST  =  131.24  +  .113768  PROC 
In  this  case  the  large  discrepency  between  the  thesis  estimate 
and  PME  107X  estimate  is  very  easy  to  explain.   First  of  all, 
the  equipment  is  to  be  installed  by  the  manufacturer  while 
the  ships  are  dockside  at  a  Navy  base  rather  than  the  usual 
practice  of  having  it  installed  by  shipyard  personnel  in  Naval 
Shipyards.   Secondly,  as  can  be  seen  from  the  last  section, 
the  installation  CER,  although  seemingly  the  best  available 
from  the  data,  is  not  particularly  good. 

As  can  be  seen  from  the  cost  estimates,  the  CER  developed 
in  this  thesis  predicts  that  DTP-3  installation  cost  will  be 
less  than  either  DTP-1,  or  DTP-2;  however,  this  could  not  be 
correct  since  DTP-3  includes  DTP-1  and  DTP-2.   This  demonstrates 
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one  of  the  problems  that  occur  when  one  attempts  to  predict 
the  cost  of  a  system  which  is  different  than  the  equipment 
in  the  data  base.   Since  the  active  coefficient  has  a 
negative  sign  with  all  other  variables  in  the  CER  held 
constant,  the  active  equipment  DTP-3,  installation  cost 
estimate  is  less. 

D.   SUMMARY  AND  CONCLUSIONS 

Cost  Estimating  Relationships  are  only  as  good  as  the 
data  base  from  which  they  are  derived.   When  applying  them, 
one  must  be  aware  of  whether  the  proposed  system  is  within 
or  is  an  extension  beyond  the  data  base.   The  larger  the  step 
beyond  the  data  base,  the  more  uncertain  the  results. 

In  the  case  of  the  DTP  EW  program,  many  things  are 
different  than  past  equipments,  consequently  the  need  for 
more  detailed  costing  is  obvious.   Installation  costs,  in 
particular,  require  a  very  detailed  study  since  the  whole 
idea  of  contractor  installed  equipment  is  new  in  EW  equipment 

This  thesis  estimated  the  costs  development,  first  unit 
procurement  and  installation  costs  for  the  DTP  EW  system. 
The  cost  estimates  for  development  cost,  although  lower  than 
those  predicted  by  the  program  manager,  are  acceptable  when 
one  understands  the  difference  between  development  of 
equipment  in  the  past  and  the  DTP  system.   The  procurement 
cost  estimated  by  this  thesis  is  consistantly  $600,000  above 
the  program  manager  estimates.   The  difference  can  only  be 
explained  by  the  fact  that  the  prices  were  derived  from 
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different  estimators,   In  the  case  of  installation  cost 
estimates,  the  thesis  CER  estimates  consistantly  above  program 
manager's  estimates,  however  since  the  new  equipment  is  to 
be  contractor  installed  vice  Naval  Shipyard  installed  the 
difference  would  be  expected. 
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APPENDIX:   DATA  BASE,  SOURCE,  AND  ADJUSTMENTS 

The  comprehensive  data  matrix  of  surface  electronic  warfare 
equipments,  Table  1,  was  taken  from  Ref.  1.   A  considerable 
attempt  was  made  to  validate  the  particular  numbers  in  the 
matrix,  however,  it  was  found  that  a  validation  of  the  cost 
figures,  in  most  cases,  would  be  impossible  due  to  a  lack  of 
accurate  documentation  for  the  earlier  equipments.   The  Air 
Force  Electronic  Countermeasures  Directory,  Ref.  4,  was  the 
only  publication  giving  specific  parametric  information  for 
all  the  systems,  however,  both  the  weight  and  volume  figures 
differed  considerably,  in  some  cases  as  much  as  a  factor  of 
fourteen  for  WT,  and  ten  for  VOL.   The  data  used  in  this 
thesis  included  the  total  package  for  a  ship,  while  the 
figure  given  in  the  ECM  Directory  was  only  for  the  black  box, 
antenna  was  not  included  nor  were  two  units  accounted  for 
when  two  units  were  required  per  ship. 

Reference  1  stated  that  a  90%  log  linear  cumulative 
average  learning  curve  was  assumed  for  EW  equipment  and  used 
throughout  that  paper.   It  was  decided  that  first  unit  procure- 
ment cost  should  be  used,  since  different  quantities  of  each 
equipment  were  purchased.   Average  unit  costs  and  quantity 
were  assumed  to  be  correct  and  calculations  were  made  to 
insure  a  ninety  percent  average  learning  curve  was  in  fact 
used.   These  calculations  indicated  learning  curves  ranging 
from  80%  to  90%  with  the  great  majority  falling  into  the 
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84-86%  range,   The  author  of  Ref,  1  charged  with  calculations 
could  not  explain  the  discrepency  and  stated  that  it  could 
have  been  an  error  in  his  method  of  calculation.   Consequently, 
first  unit  costs  were  recalculated  using  a  90%  curve,  average 
unit  costs,  and  quantity  purchased,  except  in  the  case  of 
SLQ  22,  SLQ  23,  and  SLQ  24  for  which  quantity  purchased  was 
zero;  in  that  case  first  unit  costs  given  in  Ref.  1  were  used. 
The  author  decided  on  1969  vice  1900  as  a  base  year  and 
transformed  the  YR  variable  to  conform. 

It  was  felt  that  a  measure  of  complexity  should  be 
included  as  a  possible  independent  variable.   Mr.  John  P, 
Obrien,  an  Electronics  Engineer  in  the  Design-to-Price  Office 
of  Navy  Electronic  Systems  Command  PME  107X  gave  his  expert 
opinion  of  each  of  the  equipments  in  the  data  base  as  well 
as  those  three  under  development  by  Raytheon  and  Hughes. 
Each  piece  of  equipment  was  given  numerical  value  of  complexity 
ranging  from  1,  simple,  through  2,  moderately  complex,  to  3 
very  complex.   His  evaluation  was  validated  by  other  engineers 
in  his  office. 

Since  the  data  base  contained  both  passive  and  active 
equipments,  a  dummy  variable  with  a  value  of  one  for  active 
and  zero  for  passive  equipment  was  included.   The  only  other 
possible  difference  between  the  equipment  that  it  was  felt 
could  have  some  influence  cost  was  the  size  ship  the  particular 
equipment  was  designed  for.   Two  dummy  variables  were  used 
to  take  this  difference  into  account.   The  LG  variable  very 
simply  was  one  if  the  equipment  was  designed  for  use  on 
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Guided  Missile  Destroyers  or  larger  ships  and  zero  otherwise. 
Similarly  the  SM  variable  was  one  if  the  equipment  was  designed 
for  destroyer  or  smaller  ships, 

Table  2  is  a  listing  of  the  data  matrix  as  used  in  this 
thesis.   Blanks  in  the  matrix  are  data  which  is  not  available. 
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